Electronic graphic or image systems in which the painting or drawing of a color picture can be simulated, or a portion of one image can be merged into another, by electronic means are known. One such graphic system is described in our British patent number 2,089,625 and corresponding U.S. Pat. No. 4,514,818, the teachings of which are incorporated herein by reference. This system includes operator interface means which may be used by the operator to select from a range of colors and a range of intensities and to choose from a set of notional drawing implements for use in the painting or drawing.
The color is chosen by presetting digital signals which represent the components of the selected color, whilst the implement is chosen by selecting from among different implement representations, each representation representing the notional distribution of color which could be made by the implement to the picture points which it covers. The system also includes a touch tablet and stylus combination, which is usually the interface means, for generating position signals designating the position of the stylus relative to the touch tablet. For some implements, representing say paint brushes, position signals are allowed to be generated for each movement of the stylus by the distance between picture points or a similar distance, whilst for other implements, say air brushes, position signals are generated at regular time intervals, even if the stylus is held stationary on the touch tablet.
When a position signal is produced, new video signals (pixels) are derived for every picture point in the patch covered by the selected implement. A picture store is provided and each new pixel is written at the appropriate picture point in the store. Such new pixels are derived by a processing circuit in response to the selected color and the distribution of the selected implement, and in response also--as described in the aforesaid patents--to pressure applied to the stylus and to the value of the pixel previously stored at the respective picture point in the store. The user, who it is envisaged would normally be an artist lacking experience in the use of computer based systems, paints or draws by choosing a desired color and implement and then manipulating the stylus, causing the touch tablet to generate a series of position signals which define the path or positioning of the stylus. The processing circuit derives pixels for a patch of picture points in response to position signals and these pixels are written back to the picture store. To enable the artist to observe this creation, the picture stored is read repeatedly and the pixels are applied to a TV-type color monitor, so that the build-up of the picture can be observed of course such systems are not limited to TV-type formats and any suitable video format may be adopted. The system as described avoids the problem of jaggies, an unpleasant stepping appearance normally associated with lines not lying horizontally or vertically in a raster display.
It is important to the artist that the monitor should display the build-up of the picture virtually in real time, which means in practice that the processing of all the patches of pixels which may be identified by position signals in a frame period should be completed, and the pixels written in the picture store (without interfering with reading from the store to the monitor) also within a frame period, at least when the time is averaged over a few frame periods. The number of pixels to be processed is dependent on the number of picture points required per unit of picture area, that is to say on the resolution of the system; if the resolution is increased, the number of position signals generated for a given movement of stylus, or in a given time interval is also increased, as is the number of pixels in a given brush patch. Real time processing can usually be achieved in systems operating to television standards, including even the high definition standard of 1250 lines per field at 50 fields per second, or 1125 lines/60 fields. Difficulties are however encountered if the higher resolution of print quality pictures, comprising say 4000.times.5000 picture points, is demanded. Another difficulty is that TV-type monitors for displaying such resolution pictures are not readily available as articles of commerce, and would be very costly if custom built.
To reduce these problems it has previously been proposed that the painting or drawing (hereinafter simply "painting") should be performed with the resolution of commercially available TV-type monitors. One system which adopts this approach, and thus allows for the creation or modification of a picture in accordance with modifications made to a version of the picture displayed on a display screen comprises a converter for converting an initial high resolution image held in a high capacity storage device into a lower resolution representation of the image. The lower resolution image is applied to a framestore which is repeatedly read for display of the image on a color monitor. The system also includes a touch tablet and stylus combination together with a patch address generator, a pixel processing circuit and an intensity or color store, similar to those described in our abovementioned British Patent Number 2,089,625 and corresponding U.S. Pat. No. 4,514,818, by which the artist can select colors and paint on the image. A processing circuit reads image data directly from the framestore, modifies that data in accordance with data generated by the artist painting with the touch tablet, and writes the modified image data back into the framestore for display. This painting is carried out on the low resolution image so that the processing circuit can keep pace with movements of the stylus by the artist and the result can be viewed on the monitor in real time. When the artist finishes painting, either temporarily or finally, the system transfers the modified image data back to the high capacity storage device via an up converting circuit which converts the modified image back to high resolution.
While painting at low resolution allows the artist to observe the results of his action apparently in real time, the down converting of the initial image followed by up conversion inevitably produces a softening of the high resolution picture effectively reducing the resolution of the picture to the lower resolution even though it contains the number of picture points required at high resolution.
In our European patent application no. EP-A-0,202,014 we describe a system in which a high resolution image can be modified by storing high resolution image pixel data in a high resolution framestore and painting new pixel data directly into that framestore by use of a stylus and touch tablet. In order to enable the artist to see the painted modifications as they are made a converter is provided which reduces the number of pixels between the high resolution frame store and a viewing frame store, which is read repeatedly to refresh the monitor display. The converter is arranged such that either a selected area of the high resolution image is transferred to the viewing store and displayed at full resolution on the monitor, or a low resolution representation of the full image is created and fed to the viewing store for display. Whilst this system provides for real time display of the image or part of it, as it is built up in the high resolution frame store, under most circumstances large amounts of silicon memory are required and the build up of the image in the high resolution framestore may still lag behind the stylus movements.
In equipment according to our British patent no. 2,089,625, as currently manufactured and sold, additional means are provided which allow the system to be configured to perform picture composition in addition to painting. Some configurations which allow composition are described in our British patent no. 113,950 and in corresponding U.S. Pat. No. 4,602,286, the teachings of which are also incorporated herein by reference. In such configuration, storage means are provided having the capacity to store two independent pictures, termed herein as the foreground image and the background image respectively, and a further image which is termed the control image or stencil. The control image, like the foreground and background images, can be created by the artist, using the system in a painting mode. The control image usually comprises areas of pixels having a maximum value which represents opacity and a minimum value which represents transparency and is created so that in effect it masks a selected part of the foreground image while exposing the remainder. The control image is defined by eight bits and during creation can be displayed as a single color superimposed on the foreground picture. First address means are moreover provided for reading the pixels of the foreground image and the control image in parallel from the respective parts of the storage means and applying them to a linear interpolating circuit, which is rendered operational during an image preview mode and during an image composition mode. A second address means provides the pixels in the background image also to the two interpolating means concomitantly with the reading of the foreground image pixels and the control image pixels.
The first address means is connected to a manipulator which, under stylus control or the control of other user operable means, can change the order of reading the foreground pixels and control pixels relative to the background pixels so as to effect a spatial transformation of the foreground and stencil relative to the background. Whilst the system is in preview mode, the artist can manipulate the stylus or other control means to effect a series of desired transformations such as zoom, pan, scroll, rotate, and change perspective. During a frame period the foreground image pixels in transformed order and the background image pixels are applied to the input of the first interpolating means whilst the control image is applied to a control input to act as a pixel-by-pixel interpolating coefficient. The output of the interpolating means is applied to the picture monitor and displayed, but without affecting any of the pictures as stored. The interpolation has the effect of causing the display to represent the foreground image where the control image is opaque and to represent the background image where the control image is transparent, simulating the pasting or overlaying of part in the foreground image on the background. Using the stylus or control means to vary the spatial transformation, the artist can try various positions of the insert onto the background image from the foreground image until he perceives a desired transformation. Thereupon the artist, by producing a predetermined "stick" command, switches the system to the composition mode. The foreground image pixels and background image pixels are then read to the second interpolating means and interpolated under control of the control image pixels, the foreground image and control image pixels being read in the transformation order prescribed in the last try during the preview mode. In addition, in composition mode the output of the interpolating means is read back to the storage means for the background image and produces a permanent change in that image, which simulates pasting part of the foreground on the background.
The particular composition process described in the preceding paragraph is merely illustrative and many variations of the process are possible. However it is to be noted that the difficulties in achieving real time display of the artists work during composition, and in particular during preview, can be even more acute than during painting. During preview the artist in effect attaches part of the foreground image which is to be stuck down to the point of the stylus and moves it around as seen on the monitor until he finds a desired displacement. The number of pixels to be processed for each transformation may be very much larger than for any brush stamp, and the processing needed to achieve the transformation for each pixel to be transformed can be complex especially as equations for evaluating changes in pixel addresses may be involved. Therefore it can be difficult to achieve real time operation during preview and this will severely interfere with the artists creative processes. It has also been proposed to reduce the number of pixels for which processing is required by confining the processing to pixels lying within a computer-generated boundary enclosing little more than the part of the foreground image to be stuck down, but even so real time working is unlikely to be achievable in many cases at the higher resolutions envisaged.